{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use Case 9\n",
    "==========\n",
    "\n",
    "Problem Definition:\n",
    "-------------------\n",
    "\n",
    "A climate scientist wishes to analyse potential correlations between *Ozone* and *Cloud* ECVs.\n",
    "\n",
    "Required Toolbox Features:\n",
    "--------------------------\n",
    "\n",
    "* Access to and ingestion of ESA CCI Ozone and Cloud data (Atmosphere Mole Content of Ozone and Cloud Cover)\n",
    "* Geometric adjustments (coregistration)\n",
    "* Spatial (point, polygon) and temporal subsetting\n",
    "* Visualisation of time series\n",
    "* Correlation analysis, scatter-plot of correlation statistics, saving of image and correlation statistics on disk (format options)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ingest data and create datasets\n",
    "-------------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from cate.core.ds import DATA_STORE_REGISTRY\n",
    "import cate.ops as ops\n",
    "from cate.util import ConsoleMonitor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "monitor = ConsoleMonitor()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data_store = DATA_STORE_REGISTRY.get_data_store('esa_cci_odp')\n",
    "local_store = DATA_STORE_REGISTRY.get_data_store('local')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "oz_remote_sources = data_store.query('esacci.OZONE.mon.L3.NP.multi-sensor.multi-platform.MERGED.fv0002.r1')\n",
    "oz_remote_sources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "oz_remote_sources[0].make_local(local_name='ozone.mon',\n",
    "                                time_range='2007-01-01, 2007-03-31',\n",
    "                                monitor=monitor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cl_remote_sources = data_store.query('esacci.CLOUD.mon.L3C.CLD_PRODUCTS.multi-sensor.multi-platform.ATSR2-AATSR.2-0.r1')\n",
    "cl_remote_sources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cl_remote_sources[0].make_local(local_name='clouds1.mon',\n",
    "                                time_range='2007-01-01, 2007-03-31',\n",
    "                                monitor=monitor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "local_store"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "cc = ops.open_dataset('local.clouds1.mon')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "oz = ops.open_dataset('local.ozone.mon')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "oz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter datasets to select the desired variables\n",
    "-----------------------------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cc_tot = ops.select_var(cc, 'cfc')\n",
    "oz_tot = ops.select_var(oz, 'O3_du_tot')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "oz_tot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the first time slice of the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "ops.plot_map(oz_tot, var='O3_du_tot', time='2007-01-01', file='/home/ccitbx/Desktop/fig1.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.plot_map(cc_tot, var='cfc', time='2007-01-01', file='/home/ccitbx/Desktop/fig2.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Co-register datasets by resampling\n",
    "----------------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(cc_tot['cfc'].shape)\n",
    "print(oz_tot['O3_du_tot'].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cc_tot_res = ops.coregister(oz_tot, cc_tot)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(cc_tot_res['cfc'].shape)\n",
    "print(oz_tot['O3_du_tot'].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.plot_map(cc_tot_res, var='cfc', time='2007-01', file='/home/ccitbx/Desktop/fig3.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Select the desired spatial region\n",
    "---------------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "africa = '-20.0, -40.0, 60.0, 40.0'\n",
    "# 'lon_min, lat_min, lon_max, lat_max'\n",
    "cc_tot_africa = ops.subset_spatial(cc_tot_res, africa)\n",
    "oz_tot_africa = ops.subset_spatial(oz_tot, africa)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.plot_map(cc_tot_africa, var='cfc', time='2007-01-01', file='/home/ccitbx/Desktop/fig4.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.plot_map(cc_tot_africa, var='cfc', time='2007-01-01', \n",
    "                 region=africa, file='/home/ccitbx/Desktop/fig5.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Select the desired temporal region\n",
    "----------------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cc_tot_janmar = ops.subset_temporal(cc_tot_africa, '2007-01-01, 2007-03-31')\n",
    "oz_tot_janmar = ops.subset_temporal(oz_tot_africa, '2007-01-01, 2007-03-31')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(cc_tot_janmar.time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(oz_tot_janmar.time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Retrieve and plot timeseries\n",
    "----------------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "oz_ts_point = ops.tseries_point(oz_tot_janmar, point='50, 50')\n",
    "cc_ts_point = ops.tseries_point(cc_tot_janmar, point='50, 50')\n",
    "oz_ts_mean = ops.tseries_mean(oz_tot_janmar, var='O3_du_tot')\n",
    "cc_ts_mean = ops.tseries_mean(cc_tot_janmar, var='cfc')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(oz_ts_mean)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(cc_ts_mean)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.plot(cc_ts_mean, 'cfc', file='/home/ccitbx/Desktop/fig6.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.plot(oz_ts_mean, 'O3_du_tot', file='/home/ccitbx/Desktop/fig7.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Do correlation analysis\n",
    "-----------------------"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "correlation = ops.pearson_correlation_scalar(cc_ts_mean, oz_ts_mean, 'cfc', 'O3_du_tot')\n",
    "correlation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.write_text(correlation, file='/home/ccitbx/Desktop/corr.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "correlation = ops.pearson_correlation(cc_tot_janmar, oz_tot_janmar, 'cfc', 'O3_du_tot')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "correlation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.write_netcdf4(correlation, file='/home/ccitbx/Desktop/corr.nc')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ops.plot_map(correlation, var='corr_coef', \n",
    "             region=africa, file='/home/ccitbx/Desktop/fig8.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    ""
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3.0
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}